Mining the World Wide Web

The World Wide Web has become, over the last years, a major source of information, and at the same time a significant platform for commerce. Both aspects make it an interesting target for data mining applications. In this survey, we will discuss different facets of data mining on the web, and illustrate its methods by typical application areas. These areas will be highlighted in more detail in the subsequent contributions to this special issue of the KI Journal on Web Mining. As internet based applications become more and more intertwined, we will equally consider related domains like email and newsgroups here. The contributions of the special issue indicate new trends in web mining research. Although not specifically requested in the call for papers, most of them focus on one of two issues: the detection of upcoming topics and trends, or the detection and support of online communities. We discuss in this paper that the emergence of these application domains goes together with two technical developments: the Semantic Web for explicitly representing knowledge in the web, and the Web 2.0 as an effort for facilitating user participation in the web. We will argue that the convergence of these two areas – one being an academic, top-down and the other a grass-roots, bottom-up approach – will be a major research challenge for the next years, where web mining will play a significant role.